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1.  INTRODUCTION  vd-lL  f _ 

In  domains  ranging  from  neurophysiology  to  computational  modeling,  recent  work  on 
visual-information  processing  indicates  that  even  a  single  act  of  seeing  entails  the  coding  and 
recoding  of  the  visual  stimulus  through  multiple  levels  of  representation  (e.g.,  Marr,  1982; 
Schiller,  1986).  Within  cognitive  psychology,  the  attempt  to  characterize  these  levels  of 
representation  has  focused  primarily  on  variants  of  the  following  task:  an  array  of  letters  or 
digits  is  briefly  presented  and  the  subject  attempts  to  identify  and  remember  the  characters  it 
contains.  Even  in  this  highly  constrained  and  long-studied  task,  there  remains  uncertainty  and 
controversy  about  how  many  different  representations  exist,  what  their  properties  are,  how  long 
each  can  be  maintained,  and  how  the  transformation  from  one  to  another  is  accomplished  (e.g., 
see  Coltheart,  1984).  The  present  research  had  two  broad  aims:  First,  to  better  understand 
visual  representation  within  the  conventional  cognitive  approach,  by  focusing  on  the 
computational  characteristics  of  the  internal  representations  of  character  arrays,  rather  than  on 
their  informational  content  as  most  previous  studies  have  done.  Second,  to  begin  to  extend  this 
approach  to  the  processing  of  visual  stimuli  other  than  alphanumeric  characters. 

2-  SHORT-TERM  DYNAMICS  OF  VISUAL  REPRESENTATION:  BACKGROUND 

AND  PREVIOUS  RESEARCH 

Simon  (1978)  has  distinguished  two  ways  of  characterizing  mental  representations,  in  terms 
of  their  informational  content,  and  in  terms  of  their  computational  properties.  In  Simon’s 
terms,  the  traditional  cognitive  approach  to  early  visual  processing  is  concerned  primarily  with 
informational  aspects  of  the  representation,  assessing  the  amount  of  information  it  contains  as  a 
function  of  its  ’age’  (in  msec),  with  accuracy  of  retrieval  as  the  principal  measure.  Examples 
include  the  classic  studies  that  gave  rise  to  the  concept  of  iconic  memory  (Averbach  &  Coriell, 

1961;  Sperling,  1960)  and  that  revealed  a  progressive  loss  of  information  from  large  arrays.  In 
contrast  to  this  approach,  my  colleagues  and  I  have  developed  a  new  set  of  experimental 
paradigms  that  allow  us  to  focus  on  the  computational  characteristics  of  visual  information 
processing:  How  information  in  the  visual  system  is  represented,  maintained,  transformed,  and 
extracted  during  the  processing  of  arrays.  In  these  paradigms  the  arrays  viewed  by  subjects  are 
sufficiently  small  such  that  performance  is  relatively  accurate  even  after  a  long  delay.  The 
subject’s  internal  representation  of  a  brief  display  is  probed  in  different  ways,  with  instructions 
to  respond  as  rapidly  as  possible  while  maintaining  accuracy;  the  principal  measure  in  this 
approach  is  latency  of  retrieval1  Conditions  associated  with  high  accuracy  characterize  many 
real-world  situations  and  are  likely  to  call  upon  mechanisms  different  from  those  used  in  the 
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1.  Despite  their  usefulness  in  the  study  of  informational  aspects  of  visual  representation,  acctncy  measures  may 
be  inferior  to  latency  measures  as  a  basis  for  inference  about  computational  aspects.  P ashler  (1984,  p.  432),  e.g., 
has  made  this  argument  in  his  review  of  attempts  to  decide  between  "early  selection’  and  *1118  selection" 
theories  of  attention  in  the  processing  of  visual  arrays. 
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traditional  overload  conditions.  Whereas  the  central  issue  in  the  traditional  approach  to  early 
visual  processing  has  been  the  passive  persistence  of  a  visual  image  after  display  termination, 
our  work  with  displays  that  do  not  exceed  the  system’s  capacity  has  revealed  processes  of 
active  transformation  or  encoding  of  stimulus  representations,  which  alter  the  mechanisms  by 
which  information  is  extracted.  Briefly,  the  paradigms  we  have  used  include: 

(a)  Spatial  Probe  Paradigm.  The  probe,  usually  a  visual  marker,  specifies  location;  the 
response  is  to  name  the  element  that  is  or  was  in  that  location. 

(b)  Identity  Probe  Paradigm.  The  probe  is  usually  the  spoken  name  of  a  target  element;  the 
response  is  to  name  the  element  in  the  array  to  the  right  of  the  target 

(c)  Probed  Reciting  Paradigm.  The  probe,  a  tone  burst  specifies  direction;  the  response  is  to 
recite  the  entire  array  in  that  direction. 

(d)  Location-Specific  Matching  Paradigm.  The  probe  is  a  test  element  that  also  marks  a  target 
location;  the  response  is  "yes"  or  "no",  depending  on  whether  target  and  test  elements  match. 

When  the  present  contract  began  we  had  collected  only  preliminary  data  in  paradigm  (d);  Our 
work  during  the  contract  period  concentrated  almost  entirely  on  paradigms  (a)  and  (d). 

Other  critical  ingredients  of  the  methods  we  had  developed  at  the  start  of  the  contract 
period  are  investigation  of  performance  at  various  probe  delays  up  to  about  2  seconds,  variation 
in  the  number  of  displayed  characters  or  other  elements  (array  size),  and  statistical  control  of 
retinal  eccentricity.  Variation  of  array  size,  s,  is  crucial  for  two  principal  reasons:  First,  the 
effect  of  probe  delay  on  reaction  time  (RT)  interacts  strongly  with  array  size,  sometimes 
actually  differing  in  direction  for  array  sizes  within  a  small  range.  Second,  characteristics  of  the 
functions  {RT(s4)}  that  relate  mean  RT  to  array  size  at  different  probe  delays,  d.  lead  to  a 
decomposition  of  the  effects  of  delay  that  permits  powerful  inferences  —  inferences  that  could 
not  be  supported  by  the  study  of  performance  with  a  single  array  size.2  During  the  period 
covered  by  the  present  report  we  developed  multiple-regression  techniques  to  aid  in  data 
description,  and  robust  fitting  procedures  to  deal  with  the  high  tails  of  reaction-time  (RT) 
distributions;  we  also  integrated  an  infra-red  limbus  eye-tracker  into  our  experimental 
procedure  to  better  control  eye  position.  We  have  used  the  RT(s)  function  and  changes  in 
RT(s)  with  probe  delay  to  infer  properties  of  the  internal  representation  and  of  the 
transformations  it  undergoes.  In  our  work  thus  far  RT(s)  changes  rapidly  with  delay;  moreover, 
in  three  of  the  paradigms  it  is  either  flat  or  strikingly  linear  at  all  delays,  and  hence  can  be 
fully  characterized  by  its  changing  slope  and  intercept 

What  do  our  findings  imply  for  the  nature  of  early  visual  representations  and  the  processes 
that  generate  and  transform  them?  We  have  discovered,  for  example,  that  the  earliest 
representation  is  distinguished  by  a  property  of  direct  access  by  spatial  position  (search  of 
elements  in  other  positions  is  not  necessary  to  retrieve  an  element  whose  position  is  specified), 
and  also  by  directional  symmetry  (the  initial  representation  of  a  row  of  characters  can  be 
scanned  in  either  direction).  Given  plausible  retrieval  operators,  these  properties  are  consistent 
with  an  array-format  representation  (cf.  Marr,  1982).  As  the  representation  is  transformed,  both 
of  these  properties  rapidly  disappear  The  patterns  of  retrieval  times  to  date  provide  converging 
evidence  for  a  dramatic  transformation  of  the  internal  representation  of  a  character  array  that  is 

2.  Another  line  of  research  (initiated  by  Posner  et  si.)  in  which  latency  methods  have  been  used  to  study  the 

nature  of  visual  representation  as  a  function  of  storage  time,  has  been  limited  principally  to  single  characters  — 

i.e.,  arrays  of  size  one.  See,  e.g.,  Posner  A  Keele,  1967. 


complete  in  about  0.7  seconds.  The  linearity  of  the  RT(s)  functions  described  above  may 
indicate  that  insofar  as  there  is  any  effect  of  array  size  in  those  paradigms,  it  reflects  a  process 
of  serial  search,  whose  mean  rate  changes  systematically  as  the  representation  is  transformed. 


3.  SYNOPSIS  OF  THE  PRINCIPAL  RESEARCH  SUPPORTED  BY  THE  CONTRACT 

NOTE:  Reports  containing  additional  details  are  referenced  by  numbers  in  square  brackets, 
and  listed  in  the  final  section  of  references. 

3.1  Technical  Advances  (3,61 

Our  substantive  findings  were  made  possible  by  some  of  the  following  technical  advances. 

(1)  To  help  overcome  unavoidable  nonorthogonalities  in  experimental  design  we  devised  a 
multiple-regression  method  to  provide  descriptions  of  our  data;  to  deal  with  the  variability 
induced  by  the  high  tails  of  RT  distributions  we  fitted  our  multiple-regression  models  by  a 
robust  method.  Because  robust  regression  has  seldom  been  used  in  such  applications,  we 
performed  a  careful  assessment,  comparing  it  to  ordinary  least  squares  with  respect  to  several 
criteria  that  we  developed. 

(2)  The  linear  function  RT(s)  that  relates  mean  RT  to  array  size  can  be  described  by  slope  and 
intercept  parameters.  We  developed  new  rationales  for  the  choice  of  intercept. 

(3)  We  found  a  way  to  compare  the  time  courses  of  the  changes  with  probe  delay  in  slope  and 
intercept  parameters  from  die  same  paradigm,  and  also  the  slopes  of  corresponding  functions 
from  different  paradigms. 

(4)  Three  of  four  parallel  transformation  models  that  we  developed  lead  us  to  expea  that  RTs 
at  intermediate  probe  delays  are  a  probabilistic  mixture  of  the  RTs  at  short  and  long  delays. 
We  found  a  way  to  test  this  implication  by  using  properties  of  the  RT  variance. 

(5)  We  developed  computer  simulation  tools  for  testing  serial  transformation  models. 

(6)  We  incorporated  an  infra-red  limbus  eye-tracker  into  our  e  xpe  riment-  running  program  to 
measure  and  control  eye  position,  arranged  such  that  a  trial  is  interrupted  and  feedback  given  if 
the  eyes  move  more  than  an  allowed  amount.  We  ran  one  full  experiment  successfully  with  this 
augmentation,  using  the  location-specific  matching  paradigm. 

3.2  Principal  Phenomena  in  the  Spatial  Probe  Paradigm  [2^] 

At  all  probe  delays  the  effect  of  array  size  on  mean  RT  to  name  the  element  in  the  marked 
location,  described  by  the  RT(s)  function,  is  approximately  linear  for  2  <,  s  <  6.  and  can  thus 
be  indexed  by  the  slope  of  that  function.  When  the  probe  appears  early  (50  msec  before  array 
onset)  the  slope  is  virtually  zero  (Faa  1).  As  the  probe  is  delayed,  the  slope  grows  and  the 
intercept  shrinks  (Faa  2),  with  asymptote  reached  at  a  probe  delay  of  about  0.7  sec. 

33  Comparison  of  Visual  and  Tactile  Probes  (241 

Our  initial  work  during  the  contract  period  was  completion  and  analysis  of  a  large 
experiment  (about  36,000  observations)  in  which  we  examined  performance  with  visual  and 
tactile  probes  at  six  probe  delays,  and  which  had  been  conducted  to  help  discriminate  among 
three  possible  explanations  of  Faa  1: 

(la)  Direct  Access  by  Spatial  Position. 

Early  in  the  life  of  the  memory  of  a  display,  no  search  over  other  elements  is  needed  to 
retrieve  information  about  a  particular  element  when  that  element  is  specified  by  its  spatial 
positioa 

(lb)  Marker-Induced  Distinctiveness. 

When  the  displays  of  visual  marker  and  array  are  in  sufficiently  close  temporal  proximity,  an 


internal  representation  is  formed  of  their  superimposed  images.  The  marked  digit  then  has  the 
marker  integrated  with  it,  making  it  a  highly  distinctive  pattern.  Search  for  such  a  distinctive 
pattern  among  plain  digits  is  very  rapid,  so  the  effect  of  display  size  is  negligible. 

(lc)  Marker-Induced  Shift  of  Visual  Attention. 

The  visual  marker's  abrupt  onset  automatically  causes  visual  attention  to  shift  to  its  locus.  If  an 
array  element  is  displayed  at  that  locus  during  the  period  in  which  attention  is  concentrated 
there,  then  the  element  in  that  locus  is  the  first  one  examined  in  any  search.  If  we  assume  that 
the  search  for  a  location  under  these  conditions  is  self-term  mating,  then  the  search  ends  where 
it  began,  so  the  size  of  the  array  has  no  effect 

Explanations  (lb)  and  (lc)  depend  on  the  probe  being  visual,  hence  the  comparison  of 
tactile  with  visual  probes  to  test  them.  Performances  with  the  two  kinds  of  probe  were 
remarkably  similar  Facts  1  and  2  hold  for  both,  and  parameters  are  virtually  identical.  Tactile 
markers  convey  information,  but  they  are  unlikely  to  add  visual  distinctiveness  to  a  digit,  or  to 
automatically  cause  a  shift  of  visual  attention.  For  Fact  1,  this  leaves  Explanation  (la)  —  the 
direct-access  property. 

Similarity  of  performance  with  visual  and  tactile  markers  also  allows  us  to  reject  a  potential 
explanation  of  Fact  2: 

(2a)  Change  from  Retinotopic  to  Spatiotopic  Coordinates. 

It  has  been  proposed  that  the  internal  representation  of  visual  information  changes  over  time 
from  an  initial  form,  referred  to  retinal  coordinates,  to  a  later  form,  referred  to  extra-retinal 
coordinates.  (See,  e.g.,  Breitmeyer,  Kropfl,  &  Julesz,  1982.)  If  so,  it  seems  reasonable  that 
initially  (when  we  find  direct  access)  less  "computation"  would  be  required  to  determine  the 
spatial  relationship  between  the  constituents  of  visual  displays,  even  if  nonsimultaneous:  Their 
locations  can  presumably  be  "directly"  compared.  Later  (when  we  find  that  search  is 
necessary)  both  must  be  referred  to  an  extra-retinal  coordinate  system.  Because  the  mapping 
from  tactile  marker  to  visual  array  would  have  to  be  "indirect"  at  all  delays,  the  similarity  of 
our  findings  for  tactile  and  visual  markers  argues  against  this  explanation  of  Fact  1. 

3.4  The  Roles  of  Uncertainty  about  Position,  Target,  and  Response  in  Information  Retrieval  [4] 

In  a  pair  of  experiments  we  assessed  the  roles  of  uncertainty  about  position,  target,  and 
response,  and  changes  in  such  uncertainty  with  probe  delay,  in  producing  the  dramatic  effects 
of  storage  time  on  the  retrieval  of  visual  information  specified  by  location.  While  an  array 
resides  in  memory,  information  of  three  different  kinds  may  accumulate  that  might  shorten  the 
RT  to  the  probe  and  that  might  especially  benefit  small  arrays,  thus  inducing  an  increase  in  the 
slope  of  the  RT(s)  function  (Fact  2).  We  thus  have  three  additional  candidates  for  explanation 
of  Fact  2: 

(2b)  Reduction  of  Spatial  Uncertainty. 

The  set  of  possible  probe  locations  is  specified  by  the  set  of  occupied  array  locations.  Once 
the  array  has  been  presented,  this  set  of  locations  can  be  discriminated,  causing  the  spatial 
uncertainty  of  the  probe  to  be  reduced,  with  more  reduction  achieved  for  smaller  arrays.  If 
spatial  uncertainty  influences  the  time  to  discriminate  probe  location  in  this  situation,  the  result 
would  be  an  increase  in  the  slope  of  the  RT(s)  function. 

(2c)  Reduction  of  Target-Stimulus  Uncertainty. 

(2d)  Reduction  of  Response  Uncertainty. 

While  awaiting  a  delayed  probe,  the  subject  assimilates  information  about  which  digits  it 
contains.  As  this  information  is  assimilated,  stimulus  and  response  uncertainty  are  reduced;  the 
smaller  the  array,  the  greater  the  reduction.  If  stimulus  and/or  response  uncertainty  influences 
the  time  to  decode  the  representation  of  the  target  element  and/or  generate  the  response  in  this 
situation,  the  result  would  be  an  increase  in  the  slope  of  the  RT(s)  function. 
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To  test  alternatives  2b,  2c,  and  2d,  we  constructed  two  new  experimental  procedures.  In 
one  procedure  ( advance  location  information)  we  provided  advance  information  of  the  set  of 
alternative  locations  that  might  be  queried  by  the  probe.  Insofar  as  spatial  uncertainty  plays  a 
role  in  processing  of  the  probe,  we  expected  that  this  manipulation  would  especially  benefit 
small  arrays,  and  thus  would  increase  the  slope  of  the  RT(s)  function,  supporting  explanation 
(2b).  In  a  second  procedure  ( advance  identity  information)  we  provided  advance  information 
of  the  set  of  alternative  target  stimuli  and  responses.  Insofar  as  target-element  uncertainty  and 
response  uncertainty  play  roles  in  generation  of  the  response,  we  expected  that  this 
manipulation  would  also  be  especially  beneficial  for  small  arrays,  and  thus  would  also  increase 
the  slope  of  the  RT(s)  function,  supporting  explanations  (2c)  and/or  (2d). 

In  sharp  contrast  to  these  expectations,  we  found  no  effect  of  our  manipulations  on  the 
slope  of  the  RT(s)  function.  This  reduces  the  likelihood  that  the  dramatic  increase  in  slope 
with  probe  delay  is  associated  either  with  operations  that  precede  access  to  the  internal 
representation  of  the  array  (processing  of  the  probe),  or  with  operations  that  follow  such  access 
(generation  of  the  response).  By  excluding  these  alternative  accounts  of  the  increase  in  slope 
with  probe  delay,  we  have  strengthened  the  argument  for  a  fourth  explanation  of  Fact  2: 

(2e)  Loss  of  the  Direct-Access  Property. 

The  property  of  direct  access  by  spatial  position  is  lost,  as  an  initial  random-access  memory 
(array-format  representation)  is  transformed  into  a  sequential-access  memory. 

3i  The  effect  of  Memory  Load  on  the  Time  to  Name  an  Element  Indicated  by  a  Spatial  Probe 

If  transformed  array  elements  load  memory  more  than  untransformed  ones,  then  memory 
load  would  increase  with  probe  delay.  We  ran  five  experiments  to  test  the  extent  to  which 
such  an  increasing  memory  load  contributes  to  the  increasing  effect  of  array  size  with  delay. 
All  except  the  last  experiment  had  flaws.  On  the  critical  trials  in  this  last  experiment  we 
presented  two  arrays  successively,  the  second  of  size  5=6,  in  which  each  element  in  the  first 
was  also  present  at  the  same  location  in  die  second.  Only  a  subset  of  the  elements  of  the 
second  array  appeared  in  the  first  Thus  we  were  able  to  (a)  vary  the  size  of  the  ostensible 
memory  load  (determined  by  the  size  of  the  initially-presented  subset),  at  the  same  time  as  we 
were  able  to  (b)  use  a  location  probe  to  designate  an  element  that  was  present  only  in  the 
second  array,  to  avoid  the  possibility  that  the  reaction  time  was  determined  by  the  outcome  of 
a  race  between  two  searches,  one  applied  to  each  array.  Results  indicate  a  memory  load  effect 
of  about  13  msec/element,  only  a  small  fraction  of  the  70  msec/element  slope  generated  by 
appropriately  delayed  probes.  We  conclude  that  the  growth  of  memory  load  may  play  a  role  in 
the  increasing  slope  of  the  RT(s)  function  with  probe  delay,  but  it  is  a  minor  one;  the  bulk  of 
the  effect  appears  to  be  due  to  a  loss  of  the  direct-access  property. 

3.6  Nature  of  the  Transformation  Process  [3] 

Using  data  from  the  spatial-probe  paradigm  we  investigated  both  serial  and  parallel  models 
of  the  transformation  process.  According  to  serial  models,  the  transformation  is  applied 
successively  to  the  elements  (digits)  in  the  array  until  all  elements  have  been  transformed.  It 
follows,  intuitively,  that  because  larger  arrays  take  more  time  to  transform,  mean  RT  will  reach 
asymptote  later  for  larger  arrays,  so  that  if  the  RT(s)  function  is  linear  at  long  probe  delays,  it 
will  be  concave  down  at  intermediate  delays.  Computer  simulations  of  six  specific  realizations 
of  the  serial  model  confirmed  the  intuition:  In  contrast  to  our  data,  the  fitted  functions  at 
intermediate  probe  delays  were  somewhat  nonlinear.  However,  for  all  model  variants  except 
those  in  which  the  time  to  transform  an  element  had  zero  variance,  we  were  able  to  account 
very  well  for  the  change  with  probe  delay  in  slope  and  intercept  parameters  of  RT(s).  One 
possibility,  admittedly  implausible,  is  that  artifacts  (such  as  differential  speed-accuracy 
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tradeoffs  for  different  array  sizes)  might  produce  slight  alterations  in  the  form  of  the  RT(s) 
function  and  thus  obscure  the  "true”  downward  concavity.  For  this  reason,  we  should  seek 
additional  tests  of  die  idea  that  the  transformation  process  is  sequential. 

According  to  simple  parallel  models  of  the  transformation  process,  the  transformation  is 
applied  simultaneously  to  all  elements  in  the  array  and  proceeds  at  a  rate  that  is  independent  of 
array  size.  For  three  of  the  four  variants  of  such  models  that  we  considered,  the  RT  distribution 
at  an  intermediate  delay  should  be  a  binary  probabilistic  mixture  of  the  distributions  at  longer 
and  shorter  delays.  We  devised  a  test  based  on  the  RT  variance  that  depends  on  this  property, 
subjected  our  data  to  this  test,  and  found  that  the  variance  at  intermediate  delays  was  too  small 
relative  to  what  was  expected  from  the  variances  at  longer  and  shorter  delays.  This  test  is 
attractive  because  it  depends  on  only  weak  assumptions;  it  should  be  applied  to  data  collected 
in  an  experiment  specifically  designed  for  that  purpose. 

3.7  Comparison  of  the  Time  Coarse  of  Slope  and  Intercept  Changes  with  Probe  Delay  [3,5] 

One  of  the  striking  characteristics  that  we  discovered  of  the  way  in  which  the  RT(s) 
function  changes  as  the  probe  is  delayed  is  that  the  time  course  of  the  changes  in  its  slope  and 
intercept  are  virtually  identical.  That  is,  the  proportion  of  total  change  that  is  attained  by  a 
given  delay  is  approximately  the  same  for  the  two  parameters.  Because  these  two  parameters 
are  likely  to  reflect  different  subsets  of  the  processes  that  occupy  the  time  between  probe  and 
response,  and  because  there  is  no  reason  to  expect  the  effects  of  probe  delay  to  have  the  same 
time  course  (in  the  sense  defined  above)  for  these  two  subsets,  the  property  is  surprising,  and 
places  strong  constraints  on  the  underlying  mechanisms.  One  simple  way  in  which  the 
property  could  arise  is  if  the  RT(s)  function  at  any  intermediate  delay  is  a  probabilistic  mixture 
of  the  corresponding  functions  at  two  extreme  delays,  with  a  mixing  probability  that  changes 
with  delay.  That  such  a  mixture  could  arise  from  parallel  transformation  processes,  as 
mentioned  above,  makes  further  testing  of  such  processes  of  special  interest 

We  also  compared  the  time  courses  of  change  of  the  slopes  of  two  sets  of  RT(s)  functions, 
one  set  from  the  identity-probe  paradigm  and  the  other  set  from  the  spatial-probe  paradigm. 
Here,  also,  we  found  the  time  courses  to  be  very  similar,  suggesting  that  performances  under 
the  two  paradigms  are  alternative  reflections  of  the  same  transformation  process. 

3J1  Test  of  Direct  Access  by  Color 

One  method  often  used  to  speed  access  to  information  in  visual  displays  is  to  distinguish 
elements  of  such  displays  by  color.  Under  conditions  that  were  otherwise  the  same  as  those 
we  use  in  the  spatial-probe  paradigm,  we  tested  whether  there  was  direct  access  to  an  element 
so  distinguished.  Even  when  the  color  to  be  selected  was  the  same  from  trial  to  trial  (unlike 
the  marked  location  in  the  spatial-probe  paradigm)  we  found  that  search  of  other  elements  is 
required:  we  did  not  find  evidence  for  direct  access.  It  is  possible  that  this  result  is  a 
consequence  of  the  relative  latencies  of  color  and  form  discrimination  when  the  two  attributes 
are  presented  simultaneously.  We  know  from  the  spatial-probe  paradigm  that  the  property  of 
direct  access  is  transient,  with  the  slope  of  the  RT(s)  function  rising  rapidly  with  delay  of  a 
location  marker.  Insofar  as  color  is  discriminated  with  a  delay  relative  to  form  in  our  situation, 
selection  by  a  simultaneous  color  attribute  may  be  equivalent  to  selection  by  means  of  a 
delayed  marker. 

3.9  Comparison  of  Naming  of  a  Marked  Element  with  Location-Specific  Matching  [7] 

We  ran  several  experiments  using  the  location-specific  matching  paradigm  with  arrays  of 
digits.  One  of  our  aims  was  to  compare  performance  here,  where  naming  is  not  required,  with 
performance  in  the  spatial-probe  paradigm  where  it  is.  To  what  extent  does  our  evidence  for  a 
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rapid  transformation,  or  even  the  occurrence  of  such  a  transformation,  depend  on  the 
requirement  to  name  one  of  the  array  elements?  We  found  clear  evidence  for  a  fiat  function 
(indicating  direct  access)  with  early  probes,  and  for  both  a  rise  in  slope  and  a  fall  in  intercept 
as  the  probe  was  delayed.  Ovett  naming  thus  appears  not  to  be  critical  for  these  phenomena.3 

A  second  aim  was  to  investigate  selective  access  as  a  function  of  storage  duration.  To 
investigate  direct  access  we  must  measure  the  extent  to  which  the  presence  of  elements  in 
unprobed  array  locations  influences  the  response  to  the  probe.  To  investigate  selective  access, 
on  the  other  hand,  we  must  measure  the  extent  to  which  the  content  of  unprobed  array 
locations  influences  that  response.  The  matching  procedure  permits  us  to  measure  selectivity 
of  access  by  examining  the  effects  of  a  match  of  the  probe  to  an  element  in  other  than  the 
probed  location  —  that  is,  the  effects  of  the  presence  of  a  mislocated  target.  Insofar  as  access 
is  selective,  there  should  be  no  such  effect  That  is  what  we  found,  for  early  probes.  But  just 
as  direct  access  disappears  as  the  probe  is  delayed,  so  does  selective  access  disappear.  When  it 
disappears,  the  mislocated  target  has  an  inhibitory  effect  on  negative  responses  (mismatches); 
in  a  more  recent  experiment  in  which  we  used  mislocated  targets  on  positive  trials  (where 
arrays  also  have  correctly-located  targets)  we  found  facilitation. 

In  another  experiment  we  have  been  investigating  the  effects  of  distance  within  the  array 
between  the  mislocated  target  and  the  probe,  as  a  function  of  probe  delay,  to  further  explore 
the  nature  of  the  transformation.  One  possibility,  for  example,  is  that  the  effect  of  a  mislocated 
target  "spreads"  to  more  remote  locations  as  the  transformation  proceeds.  On  the  other  hand,  if 
we  are  seeing  a  changing  mixing  probability  of  two  array  representations,  we  would  expect  no 
change  with  probe  delay  of  the  (normalized)  gradient  of  such  effects  over  distance.  As  the 
present  report  is  being  written,  this  experiment  is  in  progress. 

3.10  Location-Specific  Matching  of  Digits  versus  Unfamiliar  and  Nameless  Shapes  [7] 

In  a  final  experiment  using  the  location-specific  matching  paradigm  we  used  arrays 
composed  of  unfamiliar  shapes  with  no  learned  names.  Most  studies  of  short-term  visual 
information  processing  have  used  alphanumeric  characters;  little  attention  has  been  paid  to  the 
question  of  generality.  One  virtue  of  the  location-specific  matching  procedure  is  that  because  it 
does  not  require  naming  we  are  not  restricted  to  nameable  forms.  Our  initial  results  suggest 
that  for  early  probes  the  direct-access  property  applies  here,  also,  and,  more  surprisingly,  that 
there  is  a  similar  transformation  (albeit  with  a  longer  time  course)  to  which  arrays  of  such 
elements  are  subject  This  finding  raises  the  possibility  that  the  product  of  the  transformation 
does  not  represent  a  string  of  names,  as  one  might  otherwise  have  thought  Again,  more  work 
is  needed,  possibly  with  more  discriminable  array  elements  to  reduce  error  rate,  and  also  with 
other  changes  in  method  that  we  have  found,  reduce  variability.4 


3.  Because  the  response  set  is  independent  of  srrmy  size  in  this  paradigm,  our  finding  here  adds  to  other  evidence 
(Section  3.4)  against  any  important  role  of  differential  response  uncertainty  in  the  effects  of  probe  delay. 

4.  One  of  these  changes  is  the  incorporation  of  an  infra-red  limbus  eye-tracker  into  the  experimental  procedure  to 
measure  snd  hence  control  eye  position  at  critical  times  during  each  trial.  The  variability  reduction  achieved  by 
this  change  and  others  in  an  experiment  with  digits  u  array  elements  was,  however,  associated  with  alteration 
of  neither  our  principal  findings  nor  their  interpretation. 
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3.11  Relation  of  the  Rapid  Transformation  of  Small  Arrays  to  the  Decay  of  Iconic  Memory  of 
Large  Arrays 

With  digits  as  array  elements,  the  transformation  we  have  been  studying  seems  to  be 
complete  in  less  than  a  second,  and  proceeds  very  rapidly  during  the  first  third  of  a  second. 
This  suggests  a  relation  to  the  rapid  decline  in  storage  capacity  shown  in  iconic  memory 
experiments  with  large  arrays  and  partial  report  procedures,  shown  by  Averbach  &  Coriell 
(1961)  and  Sperling  (1960).  One  way  to  study  the  relationship  between  the  two  sets  of 
phenomena  would  be  to  use  manipulations  that  are  known  to  influence  the  duration  of  iconic 
storage  and  examine  their  effects  on  performance  with  our  procedures  and  small  arrays.  Do 
conditions  that  prolong  iconic  storage  of  large  arrays  also  increase  the  time  taken  by  the 
purported  transformation  of  small  arrays?  In  our  first  attempt  to  investigate  this  question  we 
have  been  testing  a  subject  we  discovered  accidentally  whose  iconic  memory  is  extremely 
long-lived,  as  measured  by  a  variant  of  Sperling’s  (1960)  procedure.5  We  plan  next  to  test  him 
in  our  spatial-probe  paradigm  with  small  arrays. 

3.12  Design  for  Use  of  Event-Related  Potentials  (ERPs)  to  Monitor  the  Rapid  Transformation 

Suppose  that  our  inferences  from  the  patterns  of  RT  data  in  these  experiments  are  correct, 
and  that  the  retrieval  process(es)  elicited  by  probes  at  different  delays  are  based  on  different 
representations  of  the  stored  information.  Then  the  gross  electrical  activity  of  the  brain 
associated  with  these  retrieval  processes  may  also  differ.  In  particular,  if  different  brain 
structures  underlie  the  different  representations,  and  the  retrieval  process  that  uses  a 
representation  is  localized  in  the  corresponding  brain  structure,  then  the  electrical  activity 
associated  with  retrieval  processes  at  different  delays  may  have  different  distributions  over  the 
cortex.  Thus  the  ERP  might  initially  reflect  the  "visual"  processes  subserved  by  visual 
mechanisms,  and  later  reflect  nonvisual  processes  subserved  by  other  mechanisms. 

Based  on  this  observation,  and  in  collaboration  with  M.  J.  Farah  of  Camegie-Mellon 
University,  we  have  planned  an  exploratory  study  see  whether  such  a  difference  in  the 
distribution  of  activity  can  be  detected  in  ERPs  to  probes  at  different  delays.  If  so,  then  this 
would  open  the  way  to  studying  the  time  course  of  this  change  in  the  activity  distribution,  and 
to  comparing  it  to  the  time  course  of  the  change  in  representation  inferred  from  behavioral 
measures* 

4.  CONCLUSION 

Not  only  does  research  of  this  kind  increase  our  fundamental  knowledge  of  how  people 
process  visual  information,  but  it  is  also  likely  to  have  practical  implications  in  display  design,7 
for  example,  and  in  die  understanding  of  pathologies  of  visual  processing.  The  basic  knowledge 
acquired  in  such  quantitative  behavioral  studies  of  visual  information  processing  also  provides 
crucial  guidance  for  brain  research  and  computational  models. 

3.  It  may  or  may  not  be  a  coincidence  that  this  subject  has  juvenile  diabetes. 

6.  T.  R.  B ashore  of  the  Medical  College  of  Pennsylvania  has  offered  us  the  use  of  his  ERP  laboratory  for  this 
study. 

7.  Examples  of  findings  with  practical  relevance  are  the  effectiveness  of  tactile  stimuli  and  the  superiority  of 
location  markers  over  color  differentiation  in  guiding  visual  attention,  the  opposite  effects  of  storage  time  on  the 
speed  of  access  to  small  versus  large  arrays,  and  the  long-persisting  advantages  of  simultaneous  (array)  over 
sequential  presentation  for  flexibility  of  retrieval. 
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